Systems and methods for data verification

ABSTRACT

Embodiments are directed to data verification of business or consumer data. Certain embodiments include a data verification system that receives or selects data to be verified, selects one or more verification methods to verify, update, and/or append/enhance the data. The data verification system may verify the data with one or more data verification methods, either alone or in combination. The methods may include a web-crawling verification method, an agent web verification method, a call verification method, a direct mail method, an email method, an in-person verification method, or other methods. The system has the ability to, automatically or manually, (1) blend automatic and manual segmentation of records or elements by criteria such as industry type, best times of day/month/year to verify, update, or append, cost, and level of importance (2) select the best verification processing method(s), and (3) manage the results and properly verify, update, append/enhance records.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/620,103 filed on Sep. 14, 2012, entitled “Systems and Methods forData Verification,” which is a continuation of U.S. patent applicationSer. No. 12/058,632 filed on Mar. 28, 2008, entitled “Systems andMethods for Data Verification,” which claims the benefit of priorityfrom U.S. Provisional Patent Application No. 60/921,188 filed on Mar.30, 2007, entitled “Systems and Methods for Data Verification,” theentire contents of which are each hereby incorporated herein byreference in their entirety. All publications and patent applicationsmentioned in this specification are herein incorporated by reference intheir entirety to the same extent as if each individual publication orpatent application was specifically and individually indicated to beincorporated by reference.

BACKGROUND

1. Field

The present disclosure relates to the field of data verification, moreparticularly to improved methods and systems for verifying and/orupdating data.

2. Description of the Related Art

Companies increasingly rely on internal and external data relating totheir existing or potential customers in order to make critical businessstrategy decisions. Therefore, a business need exists for systems andmethods for verifying the integrity and quality of such data.

SUMMARY OF THE DISCLOSURE

Embodiments are directed to data verification of business or consumerdata. One embodiment is a data verification system that receives orselects data to be verified, selects one or more verification methods tobe applied to the data, and verifies, updates, and/or appends/enhancesthe data.

In certain embodiments, the data verification system and/or method isconfigured to verify one or more types of data. The data may include,for example, business data, non-profit data, government data, creditdata, financial data, securities data, consumer data, individual data,pet data, web-posting data, shopping data, email data and the like. Incertain embodiments, the data verification system is configured toverify the data in one or more languages or formats or codes or thelike.

In certain embodiments, the data verification system is configured toverify data with one or more data verification methods, either alone orin combination. The methods may include a web-crawling verificationmethod, an agent web verification method, a call verification method, adirect-mail method, an email method, an in-person method, and/or othermethods. The system may utilize a lower cost method to first verify alarger amount of data records and a higher cost method to verify asmaller amount of data records. The system may segment data intomultiple segments/elements and apply a different data verificationmethod to each segment/element.

Certain embodiments of the system comprise an artificial intelligencemodule that checks the accuracy and costs of the data verificationmethods as well as other business logic (for example, best time ofday/month/year to verify) and dynamically adjusts the usage of theverification methods to meet certain pre-defined business objectives(for example, to achieve the highest accuracy at the lowest cost).

One embodiment is an automated system for verifying data comprising adata selection module configured to select at least a portion of data tobe verified, and an artificial intelligence module configured to selectone or more data verification methods, based on prior results of the useof the one or more data verification methods, from web-crawling,tele-verification, agent web verification, direct-mail verification,email verification, and in-person verification to apply to the selectedportion of the data, wherein the data is verified, updated, or appendedas a result the application of the one or more selected dataverification methods to the selected portion of the data. In anotherembodiment, the system may further comprise cost data stored on thesystem indicating the cost of the data verification methods, and theartificial intelligence module is further configured to select two dataverification methods using the cost data for the two data verificationmethods, wherein the selected data verification method with the lowercost is applied to the selected portion of the data, and the selecteddata verification method with the higher cost is applied to a subset ofthe selected portion of the data.

Another embodiment is an automated system for verifying data comprisinga data segmentation module configured to segment data to be verifiedinto a plurality of data portions, wherein each data portion comprises acharacteristic and an artificial intelligence module configured toselect a data verification method to apply to each of the plurality ofdata portions based on the characteristic of the data portion, whereindata is verified, updated, or appended as a result of the application ofthe selected data verification method.

Yet another embodiment is an automated method of data verification,comprising: selecting a portion of data to be verified; and selectingone or more data verification methods from web-crawling,tele-verification, agent web verification, direct-mail verification,email verification, and in-person verification to apply to the selectedportion of the data, based on prior results of the use of the dataverification methods, wherein the data is verified, updated, or appendedas a result the application of the one or more data verification methodsto the selected portion of the data. In another embodiment, the methodmay further comprise: storing cost data indicating the cost of dataverification methods; and the selecting one or more data verificationmethods comprises selecting two data verification methods using the costdata for the two data verification methods, wherein the selected dataverification method with the lower cost is applied to the selectedportion of the data, and the selected data verification method with thehigher cost is applied to a subset of the selected portion of the data.

Another embodiment is an automated method for verifying data comprising:segmenting data to be verified into a plurality of data portions,wherein each data portion comprises a characteristic; and selecting adata verification method to apply to each of the plurality of dataportions based on the characteristic of the data portion, wherein datais verified, updated, or appended as a result of the application of theselected data verification method.

Another embodiment is a computer program product comprising a computerusable medium having control logic stored therein for causing a computerto verify data, the control logic comprising: a first computer readableprogram code means for causing the computer to select a portion of datato be verified; and a second computer readable program code means forcausing the computer to select one or more data verification methods,based on prior results of the use of the one or more data verificationmethods, from web-crawling, tele-verification, agent web verification,direct-mail verification, email verification, and in-person verificationto apply to the selected portion of the data, wherein the data isverified, updated or appended as a result the application of the one ormore selected data verification methods to the selected portion of thedata.

Finally, one embodiment is a computer program product comprising acomputer usable medium having control logic stored therein for causing acomputer to verify data, the control logic comprising: a first computerreadable program code means for causing the computer to segment data tobe verified into a plurality of data portions, wherein each data portioncomprises a characteristic; and a second computer readable program codemeans for causing the computer to select a data verification method toapply to each of the plurality of data portion based on thecharacteristic of the data portion, wherein data is verified, updated orappended as a result of the application of the selected dataverification method.

BRIEF DESCRIPTION OF THE DRAWINGS

Specific embodiments of the invention will now be described withreference to the following drawings, which are intended to illustrateembodiments of the invention, but not limit the invention:

FIG. 1 illustrates a method of data verification according to oneembodiment;

FIG. 2 is a Venn diagram that shows how data verification methods areused in accordance to one embodiment;

FIG. 3 illustrates an example configuration of a data verificationsystem and its components according to one embodiment;

FIG. 4 is a flow diagram of an example web-crawling data verificationmethod according to one embodiment;

FIG. 5 is a flow diagram of an example tele-verification methodaccording to one embodiment;

FIG. 6 is a flow diagram showing how data verification methods areselectively applied by segments/elements according to one embodiment;

FIG. 7 is a flow diagram showing how data verification methods areselectively applied according to one embodiment; and

FIG. 8 is a flow diagram showing an example method of ranking dataverification methods according to one embodiment.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

To make critical business strategy decisions, companies often rely oninternal and external data relating to their existing or potentialcustomers. For example, some companies may conduct target marketingcampaigns directed at prospective customers exhibiting characteristicsof a certain market demographic, such as households with childrenlocated in the Midwest. Accordingly, these companies may obtain alisting of such households from a third party data provider. To ensurethat these companies are effectively targeting the desired marketsegment, it is best for the consumer data to be accurate and updated.Therefore, systems and methods are needed to check and update suchconsumer data or business data.

The same is true for any sort of business data whether generatedinternally by a company or obtained from a third party data provider.For example, a company may want to market loan products to smallbusinesses in the restaurant supply industry. However, in many instancescompanies simply have too much data or not enough resources to check andupdate the consumer or business data. Furthermore, companies often needsome or all of the data on a real-time basis, thereby preferring thatsuch data be verified and updated continuously or periodically. Forpurposes of summarizing the embodiments of the invention certainaspects, advantages and novel features of the systems and methods forverifying the integrity and quality of data will be described herein. Ofcourse, it is to be understood that not necessarily all such aspects,advantages or features will be embodied in any particular embodiment ofthe invention.

Data Verification Overview

FIG. 1 shows a method of data verification in accordance to oneembodiment. At block 110, the data verification system receives orselects data to be verified. The system may receive data from anexternal source, or may select data from a database connected to thesystem. At block 120, the system selects one or more verificationmethods to be applied to the data. At block 130, the system applies theselected one or more verification methods to verify the data. At block140, as a result of the data verification process, data may be verified,updated, and/or appended/enhanced.

In certain embodiments, a data record comprises a plurality of dataelements. For example, a data record for a company may comprise dataelements such as company name, company size, executive listings,address, and so forth. When a data record is verified, the originalcontent of the record is saved. The record may also be saved along witha time identifier to indicate the time of the verification. When a datarecord is updated, part of the data record (for example, one or moreelements) is altered. For example, the phone number of a company datarecord may be updated while the address remains unchanged. Finally, whena data record is appended or enhanced, additional information is newlyadded to the data record. For example, during the verification processit may be discovered that a company has 50 employees. The number ofemployees will thus be added as a new data element to the record for thecompany. Another example of appending or enhancing may involvediscovering a relationship between two or more data records (forexample, company A is a subsidiary of company B or individual A ismarried to individual B) and adding a relationship link between/amongthe records.

In certain embodiments, the data verification system and/or method isconfigured to verify one or more types of data. The data may include,for example, business data, non-profit data, government data, creditdata, financial data, securities data, consumer data, individual data,pet data, web-posting data, shopping data, email data and the like. Incertain embodiments, the data verification system is configured toverify the data in one or more languages or formats or codes or thelike. In certain embodiments, the data verification system is configuredto translate or standardize the data into one language, for exampleEnglish, or into a standard format or code or the like before comparingthe data to existing data or before storing the data in a database. Datamay be segmented by industries, by types, by data having common elementssuitable to be verified by a similar method, and/or by other criteria.

FIG. 2 is a Venn diagram that shows how data verification methods areused in accordance to one embodiment. A plurality of new data records240 may be received into the data verification system. The systemselects a portion of data records 250 and applies a specific dataverification method to the selected portion. In certain embodiments, thesystem chooses from a web-crawling verification method 210, an agent webverification method 220, and a call verification/tele-verificationmethod 230. In other embodiments, other verification methods such asdirect postal mail, email, or in-person verification may be used aswell. As shown in the Venn diagram, some data records (represented bydots) may be verified by multiple methods while some data records may beverified by one method. In other embodiments, some methods will beapplied to certain data elements/segments. For example, web-crawling maybe applied to verify/update/append the phone number and address elementswhile call/tele-verification may be applied to verify/update/appendother elements such as company names and owner's names.

FIG. 3 shows a data verification system in accordance to one embodiment.A data verification module 300 is a computer executable program executedon a computer server 316. The computer server 316 is in communicationwith one or more databases 318 that house data.

In certain embodiments, the data verification module 300 includes a dataselection module 302, a verification method selection/artificialintelligence module 304, and one or more data verification methodmodules. The data verification module 300 may additionally include aquality control module 312 and/or a data segmentation module 314. Thequality control module 312 performs quality control and/or rank theaccuracy of data verification methods in some embodiments. In certainembodiments, the data segmentation module 314 segments data by dataelements or segments according to certain characteristics by which dataverification method may be selected. The data verification module 300may include a cost data module 342 to monitor and/or store cost dataassociated with various data verification methods. Other embodimentscombine or separate into fewer or more modules.

The data selection module 302 selects a plurality of data records fromthe one or more database(s) 318 for the verification. In certainembodiments, the verification method selection/artificial intelligencemodule 304 selects one or more data verification modules to execute(within the dotted box in FIG. 3). The selection may be based in part ona set of business rules, some of which may maximize cost, efficiency,and accuracy of the various verification methods, or may be based inpart on the artificial intelligence module 304 monitoring and learningthe performance of various verification methods and adjusting theselection accordingly.

In the sample embodiment shown in FIG. 3, the data verification methodmodules include a tele-verification module 306, a web-crawling module308, an agent web verification module 310, and an other verificationmethods module 344, which may include methods such as direct mail,in-person verification, and so forth. In certain embodiments, thetele-verification module 306 contacts entities associated with theselected data records through communication channels 324. The entitiesmay be individuals 332, households 334, or businesses 336. The term“entity” as used herein can comprise without limitation individuals,households, businesses, non-profit organizations, governments, or thelike. Communications channels 324 may include telephone, cellphone, textmessaging, email, or the like. The web-crawling module 308 maycommunicate through a network 320 (for example, Internet, local areanetwork (LAN), wide area network (WAN), wireless network) to accessinformation located on data sources 338, which may be maintained byentities associated with the data records to be verified. Theweb-crawling module 308 may also access data located on otherthird-party data sources 340 such as commercial data sources orgovernmental data sources. Finally, the agent web verification module310 may similarly access information maintained on data sources 338,data sources 340 and/or other available sources. These verificationmethod modules verify, update, and/or amend/enhance the selected datarecords. Each of these data verification methods is further describedbelow.

Web-Crawling

In certain embodiments, the data verification system and/or methodselects at least a portion of the data to be verified, and causes theselected data to be verified by the web-crawling-type methodology orweb-crawling module 308. In certain embodiments, the terms “web-crawler”or “web-crawling” as used herein include without limitation, forexample, web spider, web robot, watching, scraping, harvesting, contentmonitoring, extraction or other like technologies. In certainembodiments, the web-crawling module 308 comprises a computer programsystem that selects certain data and compares the data with similar dataobtained from searching sources, such as via the internet.

By way of example and with reference to FIG. 3, web-crawling module 308may access data sources 338 and/or 340. For example, the web-crawlingmodule 308 may extract information from a personal page maintained by anindividual 332 on a social network site 340. Alternatively, theweb-crawling module 308 may extract information from a web site on thecompany server 338 maintained by a business 336. In addition, the webcrawling module 308 may search several internet data sources 340,including but not limited to company websites, directories, searchengines, chamber of commerce websites, yellow page directories, whitepage directories, government data, directory data, chamber of commercedata, franchise data, business financial data, business owner data,securities reports or filing data, news article data, press releasedata, online databases, and the like. The web crawling module 308 mayutilize data findings for corroboration of other data sources as well.For example, certain data elements or data records for certain industrysegments may require three sources of corroboration before a verify, anupdate, and/or an append is executed. The web-crawling results for maythus be one source of that corroboration. In one embodiment, theadministrator of the data verification system may set a system-wide orsegment-specific policy to decide how many additional sources ofcorroboration are required before the web-crawling results are used toverify, update, and/or append.

In certain embodiments, web-crawling is performed worldwide on datasources located anywhere in the world. In certain embodiments, the dataverification system compares and updates the data based on the datafound from the web-crawling. In certain embodiments, the variousinternet data sources have a pre-determined trustworthiness ranking. Incertain embodiments, the web-crawling method or program uses suchtrustworthiness ranking to determine whether to update consumer databased on the data from a particular internet data source.

In certain embodiments, the data verification system completes theweb-crawler comparing and updating automatically. In certainembodiments, automatic updating is completed where there is nodiscrepancy between the data and the data obtained from one or more ofthe several internet data sources. In certain embodiments, automaticupdating is completed when the internet data source has a hightrustworthiness ranking. In certain embodiments, the data verificationsystem presents to an operator the original data, and the data foundfrom the web-crawling. In certain embodiments, an operator of the dataverification system compares the original data with the data found fromweb-crawling and based on such a comparison, the operator directs thedata verification system to update the data with or ignore the datafound from the web-crawling. In certain embodiments, the operatormanually updates the data.

In certain embodiments, web-crawling comprises the use of watchingtechnology that monitors, for example, a certain website for updates tothe website such that the data is only updated when website updates aredetected. In certain embodiments, the data verification system updatesthe data when the system receives a notification from the watchingtechnology. In certain embodiments, updates to the website includeswithout limitation, for example, changes in text, images or otherinformation provided on the website; or increases/decreases in: trafficto the website, number of unique visitors to the website, purchasestransacted on the website, average user duration on the website or anyother website metrics or analytics; or changes in related business orindustry trends, financial market valuations or any other businessintelligence indicator. In certain embodiments, the website metrics oranalytics is provided by third party providers including withoutlimitation, for example, Google Inc, onestat.com, or the like.

In certain embodiments, the web-crawler automatically determines whetherto update the data based on a date and time comparison, wherein, forexample, the web-crawler updates the existing data with the new datafound on the internet if the internet data comprises a more recentcreation date and/or time. In certain embodiments, the web-crawlerdetermines the creation date and time of the newly found internet databy analyzing the date stamp information stored on the webpage or in thecode of the webpage. In certain embodiments, the web-crawler or othersystem is configured to update the existing data with newly found data,and store in a database the corresponding stamp date and/or timeinformation related to the newly found data. In certain embodiments, thestored stamp date and/or time data is compared with newly discoveredinternet data to determine whether such internet data was more recentlycreated. In addition, to websites, the web-crawler may monitor a varietyof systems, data sets, and applications, including a network ofcomputers, application data, database data, and so forth. In certainembodiment embodiments, some or all of the web-crawling may be performedby a third party, such as, for example 365 Media, Velocityscape, and thelike. Other embodiments of the foregoing will be apparent to those ofordinary skill in the art from the disclosure herein.

Agent Web Verification

In certain embodiments, the data verification system and/or methodselects at least a portion of the data to be verified, and causes theselected data to be verified by the agent web verification module 310.FIG. 4 shows one embodiment of the agent web verification method. Atblock 410, at least one agent, preferably multiple agents, review theselected data. At block 420, the agent(s) locate the data or similardata available on the internet. At block 430, the agent(s) compare theselected data to the data found on the internet in order to verify thequality of the selected data (for example, completeness, accuracy, andso forth). At block 440, the agent(s) determine whether to verify,update, and/or append the selected data based on the data found on theinternet. If so, at block 450, the agent(s) update the selected data.Otherwise, at block 460, the agents ignore the located data.

In certain embodiments, agents are located at a central location whereinthey have access to the data verification system via one or more networkconnections, such as, for example, a local area network (LAN) connectionor the like. In certain embodiments, agents are located at distributedor multiple locations wherein the agents have access to the dataverification system via one or more network connections, such as, forexample, a wide area network (WAN) connection or the like.

Call/Tele-verification

In certain embodiments, the data verification system and/or methodselects at least a portion of the data to be verified, and causes theselected data to be verified by a call verification methodology. FIG. 5shows one embodiment of the call verification method, which may beperformed by the tele-verification module 306. At block 510, at leastone caller, preferably multiple callers, review the selected data. Thecaller may be a human or may be an automated calling program executed ona computer or an electronic device. At block 520, the caller(s) contactthe business(es), the individual(s), the household(s) or other entitiesassociated with the data in order to verify, update, append/enhance thedata. At block 530, the caller(s) may verify, update, append/enhance thedata based on the results obtained at block 520.

In certain embodiments, the callers use predefined scripts and/orcustomized scripts in making their calls. Customized scripts may be usedfor special industry segments. The scripts are configured to increasethe number of data points that can be verified and/or to maintain afavorable call experience for the recipient of the call. In certainembodiments, the callers may be monitored by monitoring agents who canprovide feedback to the callers to improve future call experiences.

By way of example and with reference to FIG. 3, the tele-verificationmodule 306 or callers may contact individuals 332, households 334, andbusinesses 336 by various communication channels 324, includingtelephoning, emailing, mailing, internet calling, text-messaging,instant messaging, video messaging, voice-mailing, faxing or the like.

In certain embodiments, the callers determine whether to update the datawith the information received from the contacting or ignore theinformation received from the contacting. The callers may be located ata central location wherein they have access to the data verificationsystem via a network or the like, or they may be located at multiplelocations, for example, a satellite office or the caller's home, whereinthe caller has access to the data verification system via a network orthe like. In certain embodiments, callers may be located at one or morecountries across the globe.

In certain embodiments, the data verification system allows the callersto perform the contacting through the network and over the internetthrough voice over internet protocol (VOIP) technology. In certainembodiments, the callers are situated or designed to work with a teamwherein the team makes calls to verify one or more selected types ofdata and/or to verify data from one or more selected data types. Theteams are segmented to better utilize the callers, and may, for example,be segmented based on strengths and/or weaknesses of the data and/or thecallers. For example, certain agents are trained to verify, update,and/or append certain data elements or data records for certain industrysegments.

In certain embodiments, some or all of the call verification maybeperformed by a third party provider such as, for example, ePerformax,eTelecare, Direct Mail, and the like. Other embodiments of any of theforegoing will be apparent to those of ordinary skill in the art fromthe disclosure herein.

Hybrid Method

In certain embodiments, the data verification system and/or method usesa combination of at least two of the verification methods describedabove (web-crawling, agent web verification, call/tele-verification,direct mail, email, in-person or other methods) to verify the data. Theverification system includes a verification method selection/artificialintelligence module 304 that selects the appropriate data verificationmethod(s). In certain embodiments, the data verification system and/ormethod selects at least a portion of the data to be verified by one ofthe foregoing verification methods, while selecting another portion ofthe data to be verified by another one of the foregoing verificationmethods.

FIG. 6 shows a sample method of applying data verification methods. Atblock 610, data to be verified is received. At block 620, a low costverification method is used to verify the data received at block 610.Then at block 630, a higher cost verification method is used to verifythe data. The higher cost verification method may be applied to all thedata that is verified by the low cost method at block 620, or may beapplied to a portion of the data that is verified by the low costmethod. The data is verified, updated, or appended/enhanced depending onthe results of the verification performed at block 620 and/or block 630.In certain embodiments, at block 650, the results of the higher costverification method applied at block 630 are sent back as feedback toimprove the results of the low cost verification method. Examples ofthis feedback may include the types of data records that can be verifiedsolely with a low cost method, or the types of data records that shouldbe verified with a higher cost method. For example, in certainembodiments, the data verification system executes a web-crawlingverification method at block 620 and then monitors or tracks the agent'sdeterminations to rank the trustworthiness of the internet data sourcesat block 630. In certain embodiments, such trustworthiness ranking datais used as feedback at block 650 to improve the accuracy of theweb-crawling module executed at block 620. It is recognized that theblocks in FIG. 6 can be rearranged to accommodate variousconfigurations, for example, a higher cost method may be used first.

Data Segmentation

Returning to FIG. 3, in certain embodiments the data verification systemincludes a data segmentation module 314 that segments data. In certainembodiments, the data verification system and/or method is configured tosegment the data to determine which data elements relate to high valuecustomers or clients. For example, the determination of whether acustomer or client is high value or otherwise is based on severalfactors, including but not limited to income, home location, net worth,credit score, and so forth. Based on the segmentation, the dataverification system may verify the segmented data relating to high valuecustomers or clients before segmented data relating to lower valuecustomers or clients. Data designated as high value or high priority maybe verified by call/tele-verification or by the verification method thatis most accurate. In certain embodiments, the data verification systemis configured to have high value or high priority data verified by acombination of the foregoing verification methods in order to doublecheck the accuracy of the data (for example, the combination shown inFIG. 6).

In certain embodiments, the data verification system and/or method usesthe data verification methodology with the highest or higher accuracy,or highest or higher ranking, as set forth above, to verify thesegmented data relating to high value customers or clients. In certainembodiments, the data verification system and/or method uses the dataverification methodology with the lowest or lower accuracy or lowest orlower ranking, as set forth above, to verify the segmented data relatingto low value customers or clients.

In certain embodiments, the data verification system and/or method usesthe data verification methodology with a medium accuracy or mediumranking to verify the segmented data relating to medium value customersor clients. In certain embodiments, the data verification system and/ormethod is configured to segment the data as soon as the data isreceived, or on a periodic basis, for example, daily, monthly, yearly orthe like. Other embodiments of any of the foregoing will be apparent tothose of ordinary skill in the art from the disclosure herein.

In certain embodiments, the data verification system performs asegmentation of the data based on related industry and/or timing, anddetermines which verification methodology is most or more accurate orleast or less expensive for collecting data related to the industryand/or timing. For example, for financial reporting data, the dataverification system, in certain embodiments, is configured to use theweb-crawling methodology to retrieve financial data from a specificfinancial data reporting website during earnings reporting season. Forexample, the web-crawling method may be configured to access 10-Kreports of companies two weeks after they are required to file thereports with the Securities and Exchange Commission. In certainembodiments, the data verification system is configured to automaticallyroute the data to be verified to the appropriate verificationmethodology, for example, call verification wherein the caller is partof a virtual call center that allows the caller to work from home.

In certain embodiments, data is segmented so that new data is verifiedby higher cost methods, and older data is verified by lower cost methodsor vice versa. In other embodiments, data is segmented to ensure propercompliance with local, state, federal, national, and/or internationallaws/regulations. For example, consumer data may be segmented to takeinto account that consumer data regulation is stricter than businessdata regulation. Data may also thus be segmented by geographic locationsto ensure proper compliance with local laws and regulations.

Artificial Intelligence

In certain embodiments, the data verification system/method includes anautomated, artificial intelligence module 304 that evolves by trackingand learning patterns of successful updates, usability, best practiceswithin segments of data, timing of year/month/day to attain bestverification/updates/appends, and so forth. The artificial intelligencemodule 304 may take into account results generated by both the qualitycontrol module 312 and the segmentation module 314. In certainembodiments, the system/method including the artificial intelligencemodule 304 has the ability to, automatically or manually, (1) blendautomatic and manual segmentation of records or elements by criteriasuch as industry type, best times of day/month/year to verify, update,and/or append, cost, and level of importance (2) select the bestverification processing method(s), and (3) manage the results andproperly verify, update, append/enhance records.

In certain embodiments, the artificial intelligence module 304 isconfigured to store and track the time period or season when aparticular verification method produces the most or more accurateresults and/or least or less expensive results and/or the most or moreefficient results. In certain embodiments, the artificial intelligencemodule 304 is configured to store industry information related to thedata such that the system is configured to determine which verificationmethod to use given a particular industry and/or season/time period.

In certain embodiments, under the hybrid method and/or thetele-verification method, the processing of data (full records orspecific elements of the records) may run through multi-tiered levels ofverification/updates/appends depending on type/segment of data in orderto gain the best data with the least related expense. With respect toFIG. 7, the sample multi-tiered method receives data to be verified atblock 710. Then data is segmented into a plurality of segments and anappropriate data verification method is determined for each segment. Inaddition, the order in which the data verification methods are appliedmay also be determined. In certain embodiments, the data verificationmethods and their order of execution may be determined by theverification method selection/artificial intelligence module 304.

At block 730, a first data verification method is applied to a firstsegment of the data. Then, at block 740, a second data verificationmethod is applied to a second segment of the data. Finally, at block750, a third data verification method is applied to a third segment ofthe data. As shown by block 760, the number of data verification methodsand the number of segments can be any number and are not limited to theexample shown in FIG. 7. In addition, the data verification method foreach segment does not have to be different, that is, the same dataverification method may be applied to multiple segments. At each blockwhere a segment is being verified, the results may be sent as feedbackto improve the learning of the verification method selection/artificialintelligence module 304. In addition, at each block where a segment isbeing verified, data may be verified, updated, or amended/enhancedaccording to the results of the data verification (block 770). In otherembodiments, the data is segmented by different elements and methods ofverification are selected based on the characteristics of the elements.

The method shown in FIG. 7 may be further illustrated by the followingexample. For data records within the Business Services industry segment,the artificial intelligence module 304 could recognize or learn fromfeedback obtained from past operations that the best way toverify/update/append this type of data records is by (1) routing themain demographic elements (name, address, phone) to a automated dialingprocess to test connectivity of phone, (2) using the web-crawlingprocess to extract on-line information or check automated postaldeliverability system for address element updates, and (3) sending thedata records to the tele-verification method for population of otherin-depth data elements. The dialing process may be a one-dial processthat puts phones on an automatic dialer during off-hours or on weekends.In one embodiment, the artificial intelligence module selects a lowercost data verification method (for example, web-crawling) for a datasegment/element that has a characteristic indicating reliable data isreadily available on-line and a higher cost data verification method(for example, agent web verification or tele-verification) for anotherdata segment/element with a characteristic indicating reliable data isnot readily available on-line.

In certain embodiments, the data verification system is configured toinclude one or more methods depending on the cost. Balancing against thecost of each verification method is the fact certain data elements orcertain industry segments are more valuable than others. For example, abusiness name element is more valuable than other details of thebusiness, and a data record in the business segment is more valuablethan a data record in the government segment. Therefore, some valuabledata elements/segments may be verified by multiple methods, including anexpensive method such as call/tele-verification, while other dataelements/segments may have a cost threshold that allow only certain dataverification methods to be used. The artificial intelligence module maytake the cost of verification and the value of the data elements and/orsegments into account when it selects the verification method. Asanother example, the web-crawling may include one or more data sourcesthat charge fees for their use such that using the web-crawling on thosedata course may exceed a threshold cost. In other embodiments, othercosts may be considered.

In certain embodiments, the data verification system is configured toprocess the data on a real-time basis. In certain embodiments, the dataverification system is configured to process the data on a batchprocessing or periodic basis.

Quality Control/Method Ranking

In certain embodiments, the data verification system is configured tocompare the accuracy of the foregoing data verification methods and rankthe methods. In certain embodiments, the results of quality control andranking performed by the quality control module 312 are sent to theartificial intelligence module 304 to assist the learning process andenhance future selection of data verification methods.

With reference to FIG. 8, the quality control module 312 may check theaccuracy of data verification methods at block 810. In certainembodiments, the quality control involves taking a sample of datarecords previous verified by a data verification method and compare themto the results of another data verification method. At block 820, thequality control module 312 is configured to dynamically rank,continuously or periodically, the foregoing data verification methodsrelative to their accuracy.

At block 830, the quality control module 312 is configured todynamically compare and/or rank, continuously or periodically, therelative expense of using the foregoing verification methods.Optionally, at block 840, the quality control module 312 is configuredto dynamically compare and/or rank, continuously or periodically, theforegoing verification methods based on other business rules.

At block 850, in certain embodiments, the quality control module 312 isconfigured to switch dynamically between the foregoing data verificationmethods. At block 860, in certain embodiments, the quality controlmodule 312 is configured to increase the use of highly ranked dataverification methods over lower ranked data verification methods basedon accuracy, expense, or other ranking criteria (for example, regulatorycompliance, timing, and so forth). In certain embodiments, at block 870,the data verification system provides reports, written or graphical orotherwise, for comparing the verification methods. The data verificationsystem may provide reports that include without limitation, for example,dashboards, scorecards, or the like.

In certain embodiments, the data verification system is configured tosignal to an operator or system administrator or project manager or thelike when a certain data verification methodology falls below a certainaccuracy level. In certain embodiments, the data verification system isconfigured to cause or start an investigation when the data verificationsystem detects that a certain data verification methodology has fallenbelow a certain accuracy threshold.

Various Embodiments of System and Method Implementations

In certain embodiments, the systems and methods for verifying andupdating data may be embodied in part or in whole in software that isrunning on a computing device. The functionality provided for in thecomponents and modules of the computing device may comprise one or morecomponents and/or modules. For example, the computing device maycomprise multiple central processing units (CPUs) and a mass storagedevice, such as may be implemented in an array of servers.

In general, the word “module,” “application”, or “engine,” as usedherein, refers to logic embodied in hardware and/or firmware, and/or toa collection of software instructions, possibly having entry and exitpoints, written in a programming language, such as, for example, Java,C, and/or C++. These may be compiled and linked into an executableprogram, installed in a dynamic link library, or may be written in aninterpreted programming language such as, for example, BASIC, Perl, orPython. It will be appreciated that modules, applications, and enginesmay be callable from others and/or from themselves, and/or may beinvoked in response to detected events or interrupts. Instructions maybe embedded in firmware, such as an EPROM.

It will be further appreciated that hardware may be comprised ofconnected logic units, such as gates and flip-flops, and/or may becomprised of programmable units, such as programmable gate arrays orprocessors. The modules, applications, and engines described herein arein certain applications preferably implemented as software modules, butmay be represented in hardware or firmware in other implementations.Generally, the modules, applications, and engines described herein referto logical modules that may be combined with other modules and/ordivided into sub-modules despite their physical organization or storage.

In some embodiments, the computing device(s) communicates with one ormore databases that store information on individuals, households, andbusinesses, including credit data and/or non-credit data. This databaseor databases may be implemented using a relational database, such asSQLite, Sybase, Oracle, CodeBase, mySQL, and Microsoft® SQL Server aswell as other types of databases such as, for example, a flat filedatabase, an entity-relationship database, and object-oriented database,and/or a record-based database.

In certain embodiments, the computing device is IBM, Macintosh, and/orLinux/Unix compatible. In another embodiment, the computing devicecomprises a server, a laptop computer, a cell phone, a Blackberry, apersonal digital assistant, a kiosk, or an audio player, for example. Incertain embodiments, the computing device includes one or more CPUs,which may each include microprocessors. The computing device may furtherinclude one or more memory devices, such as random access memory (RAM)for temporary storage of information and read only memory (ROM) forpermanent storage of information, and one or more mass storage devices,such as hard drives, diskettes, or optical media storage devices. Incertain embodiments, the modules of the computing are in communicationvia a standards based bus system, such as bus systems using PeripheralComponent Interconnect (PCI), Microchannel, SCSI, Industrial StandardArchitecture (ISA) and Extended ISA (EISA) architectures, for example.In certain embodiments, components of the computing device communicatevia a network, such as a local area network that may be secured.

The computing is generally controlled and coordinated by operatingsystem software, such as the Windows 95, Windows 98, Windows NT, Windows2000, Windows XP, Windows Vista, Linux, SunOS, Solaris, PalmOS,Blackberry OS, or other compatible operating systems. In Macintoshsystems, the operating system may be any available operating system,such as MAC OS X. In other embodiments, the computing device may becontrolled by a proprietary operating system. Conventional operatingsystems control and schedule computer processes for execution, performmemory management, provide file system, networking, and I/O services,and provide a user interface, such as a graphical user interface (GUI),among other things.

The computing device may include one or more commonly availableinput/output (I/O) devices and interfaces, such as a keyboard, mouse,touchpad, microphone, and printer. Thus, in certain embodiments thecomputing device may be controlled using the keyboard and mouse inputdevices, while in another embodiment the user may provide voice commandsto the computing device via a microphone. In certain embodiments, theI/O devices and interfaces include one or more display device, such as amonitor, that allows the visual presentation of data to a user. Moreparticularly, a display device provides for the presentation of GUIs,application software data, and multimedia presentations, for example.The computing device may also include one or more multimedia devices,such as speakers, video cards, graphics accelerators, and microphones,for example.

In certain embodiments, the I/O devices and interfaces provide acommunication interface to various external devices. For example, thecomputing device may be configured to communicate with one or morenetworks, such as any combination of one or more LANs, WANs, or theInternet, for example, via a wired, wireless, or combination of wiredand wireless, communication links. The network communicates with variouscomputing devices and/or other electronic devices via wired or wirelesscommunication links.

Although the foregoing disclosure has been described in terms of certainembodiments, other embodiments will be apparent to those of ordinaryskill in the art from the disclosure herein. Moreover, the describedembodiments have been presented by way of example only, and are notintended to limit the scope of the disclosure. Indeed, the novel methodsand systems described herein may be embodied in a variety of other formswithout departing from the spirit thereof. Accordingly, othercombinations, omissions, substitutions and modifications will beapparent to the skilled artisan in view of the disclosure herein. Forpurposes of discussing the invention, certain aspects, advantages andnovel features of the invention have been described herein. Of course,it is to be understood that not necessarily all such aspects, advantagesor features will be embodied in any particular embodiment of thedisclosure.

1. (canceled)
 2. A system comprising: an electronic data store thatstores data associated with each of a plurality of individuals; and acomputing device, comprising a physical processor, that is incommunication with the electronic data store and that is configured to:retrieve, from the electronic data store, a plurality of data elementsassociated with an individual; electronically locate informationregarding the individual on a webpage, wherein the information islocated by searching content of a plurality of webpages using at leastone of web crawling or web scraping; extract the located informationregarding the individual from the webpage; determine a creation dateassociated with the located information based on at least one (a)information on the webpage or (b) code of the webpage; perform a firstcomparison of the located information regarding the individual with atleast one of the plurality of data elements associated with theindividual retrieved from the electronic data store; perform a secondcomparison of the creation date associated with the located informationand a date stored in the electronic data store in association with theleast one of the plurality of data elements; and based at least in parton the first comparison and the second comparison, store one or moreupdated or appended data elements associated with the individual in theelectronic data store, wherein the one or more data elements are updatedor appended to include at least a portion of the located information. 3.The system of claim 2, wherein the information regarding the individualis located by at least one of an automated web spider or web robot. 4.The system of claim 2, wherein the computing device is furtherconfigured to assess trustworthiness of the webpage prior to storing theone or more updated or appended data elements.
 5. The system of claim 2,wherein the webpage comprises a page maintained by the individual inassociation with a social networking service.
 6. The system of claim 2,wherein the webpage is associated with one of a company website or agovernment website.
 7. The system of claim 2, wherein storing the one ormore updated or appended data elements associated with the individual inthe electronic data store is further based at least in part on atrustworthiness ranking associated with the webpage and atrustworthiness ranking of at least one other source of informationregarding the individual.
 8. The system of claim 2, wherein thecomputing device is further configured to, prior to storing the one ormore updated or appended data elements, verify the located informationat least in part by identifying the located information from a sourceother than the webpage.
 9. The system of claim 8, wherein verifying thelocated information comprises applying a verification method thatincludes at least one of web crawling or an automated telephone callingprogram executed by a computing system.
 10. The system of claim 9,wherein the verification method is selected based at least in part on adetermination of value associated with the located information.
 11. Thesystem of claim 10, wherein the verification method is further selectedbased at least in part on both (a) a cost associated with theverification method and (b) a reliability associated with theverification method.
 12. A computer-implemented method comprising: asimplemented by one or more computing devices configured with specificexecutable instructions, retrieving, from an electronic data store thatstores information regarding a plurality of individuals, a plurality ofdata elements associated with an individual; electronically locatinginformation regarding the individual on a webpage, wherein theinformation is located by searching content of a plurality of webpagesusing at least one of web crawling or web scraping; extracting thelocated information regarding the individual from the webpage;determining a creation date associated with the located informationbased on at least one (a) information on the webpage or (b) code of thewebpage; performing a first comparison of the located informationregarding the individual with at least one of the plurality of dataelements associated with the individual retrieved from the electronicdata store; performing a second comparison of the creation dateassociated with the located information and a date stored in theelectronic data store in association with the least one of the pluralityof data elements; and based at least in part on the first comparison andthe second comparison, storing one or more updated or appended dataelements associated with the individual in the electronic data store,wherein the one or more data elements are updated or appended to includeat least a portion of the located information.
 13. Thecomputer-implemented method of claim 12, wherein the informationregarding the individual is located by at least one of an automated webspider or web robot.
 14. The computer-implemented method of claim 12,wherein the webpage comprises a page maintained by the individual inassociation with a social networking service.
 15. Thecomputer-implemented method of claim 12, wherein the webpage isassociated with one of a company website or a government website. 16.The computer-implemented method of claim 12, wherein storing the one ormore updated or appended data elements associated with the individual inthe electronic data store is further based at least in part on atrustworthiness ranking associated with the webpage.
 17. Thecomputer-implemented method of claim 12, further comprising, prior tostoring the one or more updated or appended data elements, verifying thelocated information based on identifying the located information from asource other than the webpage
 18. A computer-readable, non-transitorystorage medium storing computer executable instructions that, whenexecuted by one or more computer systems, configure the one or morecomputer systems to perform operations comprising: retrieving, from anelectronic data store, a plurality of data elements associated with anindividual; electronically locating information regarding the individualon a webpage, wherein the information is located by searching content ofa plurality of webpages using at least one of web crawling or webscraping; extracting the located information regarding the individualfrom the webpage; performing a comparison of the located informationregarding the individual with at least one of the plurality of dataelements associated with the individual retrieved from the electronicdata store; and based at least in part on the comparison, storing one ormore updated or appended data elements associated with the individual inthe electronic data store, wherein the one or more data elements areupdated or appended to include at least a portion of the locatedinformation.